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HISTORICAL  ANTECEDENCE 

The  unprecedented  development  of  increasingly  complex  weapon  systems  during 
World  War  II  was  accompanied  by  a corresponding  need  for  training  programs  that 
could  effectively  prepare  sufficient  numbers  of  military  personnel  to  operate 
and  maintain  these  systems.  At  that  time,  however,  neither  the  psychology  of 
learning  and  performance,  nor  educational  methodology,  were  sufficiently  ad- 
vanced to  offer  any  fundamentally  new  approach  to  the  problem  of  large-scale 
training  and  little  appears  to  have  been  accomplished  during  this  period  other 
than  the  emergence  of  the  problem  Itself. 

At  the  same  time  an  even  more  basic  problem  arose  in  response  to  the 
growing  complexity  of  weapons  systems,  viz. , the  problem  of  understanding  the 
Integrated  functions  of  conceptually  difficult  systems.  An  approach  to  this 
problem  that  proved  successful  was  introduced  in  the  early  1950s  and  came  to 
be  called  "systems  analysis."  Because  this  methodology  offered  a means  by 
which  the  functional  relationships  among  component  processes  of  complex  systems 
could  be  established.  It  also  provided  an  approach  for  evaluating  the  training 
requirements  Inherent  to  these  systems.  This  new  methodology  continued  to  be 
developed  throughout  the  decade  of  the  1950s,  although  the  application  of 
systems  analysis  to  Instructional  design  of  training  programs  was  largely  re- 
garded as  a job  for  the  Instructional  specialist.  Consequently,  little  prog- 
ress was  made  toward  the  development  of  a procedural  1 zed  methodology  which 
could  be  applied  by  military  personnel.  By  the  end  of  the  decade,  however, 
this  situation  had  begun  to  change. 

Total  reliance  on  Instructional  specialists  for  training  problem  analysis 
and  program  development  was  found  to  be  undesirable.  The  cost  of  specialist 
services  was  more  expensive  than  that  of  military  personnel,  and  specialist- 
developed  programs  often  lacked  sufficient  Inputs  from  military  subject  matter 
experts  for  the  programs  to  be  realistically  valid.  Even  more  serious  concerns 
with  the  nonprocedural 1 zed  specialist  approach  stemmed  from  Inadequate  develop- 
ment of  management  and  quality  control  processes  to  assure  uniformity  and 
continuity  throughout  the  life  cycle  of  training  programs.  While  these  prob- 
lems Indicated  the  need  for  a new  methodology,  the  factors  that  had  provided 
the  basic  Impetus  for  the  original  methodology  (weapons  systems  complexity, 
training  time-cost  constraints,  trained  personnel  throughput  demands,  and 
limited  availability  of  applicable  Innovations  In  instructional  technology) 
were  now  of  even  larger  magnitude  and  they  virtually  required  a new,  more 
standardized  methodology. 

The  decade  of  the  1960s  brought  a flourish  of  methodological  activity  In 
all  of  the  armed  services,  especially  in  those  areas  where  weapon  system  oper- 
ation and  maintenance  had  reached  a level  of  complexity  that  required  special 
training.  More  than  100  technical  manuals  were  generated  during  this  period, 
each  prescribing  a highly  proceduralized  method  of  analyzing  training  problems 
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and  developing  training  programs.  A comprehensive  bibliography  of  the  litera- 
ture was  compiled  by  Montemerlo  and  Tennyson  (1976)^ • 

While  the  various  methods,  for  example.  Systems  Engineering  of  Training, 
Training  Situation  Analysis,  Developmental  Approach  to  Training,  Design  of 
Instructional  Systems,  Systems  Approach  to  Training,  etc.  differed  in  detail, 
they  all  attempted  to  Incorporate  the  systems  analysis  approach  Into  the  frame- 
work of  a standardized  set  of  procedures  which  could  be,  at  least  partially, 
applied  by  a nonspecialist  personnel.  Although  this  effort  to  achieve  some 
standardization  in  the  instructional  design  process  resulted  In  procedures 
that  were  often  so  narrow  in  scope  that  general  applicability  was  sacrificed, 
the  overall  process  Involved  In  this  effort  may  be  viewed  as  a kind  of  evolu- 
tionary development  where  success  Is  measured  In  terms  of  the  survival  of  the 
approach  rather  than  In  terms  of  a specific  outcome  from  any  particular  appli- 
cation of  It.  That  the  approach  has  been  successful  In  this  sense.  If  In  no 
other.  Is  substantiated  by  the  fact  that  It  has  persisted  into  the  latter  part 
of  the  present  decade,  and  It  Is  continuing  to  evolve  both  In  scope  and  appli- 
cability. 

An  advanced  form  of  the  systems  approach  to  training  has  been  developed 
I under  the  auspices  of  the  Naval  Air  Systems  Command  by  the  Naval  Training 

Equipment  Center  (NAVTRAEQUIPCEN).  This  comprehensive  extension  of  ISO  has 
come  to  be  known  as  the  NAVAIR/NAVTRAEQUIPCEN  model.  The  model  is  presented 
in  detail  In  a document  entitled  "Training  Requirements  for  Aviation  Weapon 
Systems"  (Specification  MIL-T-29053  draft,  October  1977).  An  expanded  treat- 

ment  of  the  model  was  prepared  by  Courseware,  Inc.  A nontechnical  description 
of  the  major  points  of  the  NAVAIR/NAVTRAEQUIPCEN  model  Is  given  in  this  report. 

EVOLUTION  OF  THE  NAVAIR/NAVTRAEQUIPCEN  MODEL 

Various  applications  of  the  Systems  Approach  to  Training  (SAT)  were  at- 
tempted by  the  Navy  during  the  1969s  and  early  1970s,  with  varying  degrees  of 
i success.  Although  the  basic  concept  appears  to  have  been  sound,  applications 

of  SAT  were  largely  Isolated  responses  to  specific  training  needs  with  little 
uniformity  from  one  application  to  another.  Thus,  there  was  no  general  mech- 
anism for  applying  SAT  In  different  training  situations,  nor  did  there  exist 
aqy  means  for  evaluating  SAT  as  a whole  by  examining  specific  applications  of 
It.  Given  this  state  of  affairs,  further  growth  In  either  the  concept  of  SAT. 
or  Its  applicability,  was  practically  precluded. 

The  Navy  needed  a system  for  developing  and  managing  its  training  programs 
which  would  be  sufficiently  general  to  assure  a wide  range  of  application  with- 
out sacrificing  the  degree  of  specif leal ity  required  for  quality  control  and 
revision.  The  needed  system  would  have  to  provide  mechanisms  for  Integrating 


TI  Montemerlo,  h.  t).  and  Tennyson,  M.  E.  (February,  1976)  "Instructional 
Systems  Development:  Conceptual  Analysis  and  Comprehensive  Bibliography," 
NAVTRAEQUIPCEN  Report  No.  IH-257. 

2.  Courseware,  Inc.  (June,  1977)  "Fleet  Aviation  Instructional  Systems 
Development  Model,"  Courseware  Technical  Report,  Vols,  I -VII. 
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the  activities  of  the  various  Naval  organizations  and  Independent  contractors 
who  contribute  resources  and  services  essential  to  training  program  development 
and  Implementation.  Also,  the  system  would  have  to  specify  procedures  for 
accomplishing  each  individual  phase  of  training  program  development  from 
Initial  problem  Identification  and  analysis  on  through  to  final  Implementation 
of  the  completed  training  program.  These  procedures  would  need  to  be  detailed 
enough,  (a)  to  permit  valid  projections  of  time,  personnel,  resources,  and 
cost  requirements  for  each  phase  In  the  development  of  the  new  program,  (b)  to 
enable  effective  management  control  of  the  time-rate  of  development  and  quality 
of  output  from  developers,  (c)  to  ensure  adequate  delineation  of  the  roles, 
responsibilities,  and  training  required  of  all  personnel  participating  In  the 
project  both  before  and  after  program  Implementation,  and  (d)  to  assure  the 
availability  of  resources  and  trained  personnel  necessary  for  a smooth  and 
effective  transition  from  the  developmental  process  to  the  actual  process  of 
managing  an  ongoing  training  program. 

In  addition,  the  new  system  should  Incorporate  those  essential  aspects  of 
the  SAT  concept  that  had  proved  to  be  successful  In  the  past,  e.g.,  task  anal- 
ysis and  selection,  ordering  of  behavioral  objectives  Into  hierarchies,  selec- 
tion of  appropriate  Instructional  methods  and  media,  and  organization  of  in- 
structional units  Into  meaningful  sequences.  But  these  Important  elements  of 
SAT  would  have  to  be  even  more  explicitly  formulated  in  the  new  system  If 
military  personnel  were  to  function  optimally  as  subject  matter  experts  under 
the  direction  of  an  instructional  design  professional,  where  this  group  would 
operate  as  a team.  Indeed,  the  primary  user  of  the  new  system  would  be  this 

, team,  and  the  system  should  be  designed  to  serve  as  a guide  for  team  develop- 

ment of  Instructional  programs. 

As  the  present  decade  reached  its  midpoint,  these  considerations  for  a new 
system  became  crystallized  Into  a formulation  designated  Instructional  Systems 
Development  (ISO),  a title  reflective  of  both  the  breadth  and  pragmatics  of 
this  new  orientation.  The  particular  formulation  that  emerged  during  this 
period  from  the  ISO  group  at  NAVTRAEQUIPCEN  - the  NAVAIR/NAVTRAEQUIPCEN 
model  — epitomized  the  many  developments  In  large-scale  Instructional  design 
that  spanned  nearly  three  decades.  Construction  of  the  NAVAIR/NAVTRAEQUIPCEN 
model  benefited  from  the  psychological  and  educational  expertise  of  several 
independent  contractors,  most  notably  Courseware,  Inc.  of  San  Diego, 

California,  Calspan  Corp. , Buffalo,  NY,  Grumman  Aerospace  Corp.,  Bethpage,  NY, 
Mathetlcs,  Inc.,  San  Diego,  California,  logicon,  San  Diego,  California,  and 
Seville  Research  Corp.,  Pensacola,  Florida. 

Already  the  NAVAIR/NAVTRAEQUIPCEN  model  has  been  widely  applied  throughout 
Naval  aviation  In  the  development  of  a variety  of  training  programs  (F-4, 

EA-6A  and  EA-6B,  A-6E,  F-14,  E-2B  and  E-2C,  SH-2F,  P-3,  S-3,  and  F-18),  and 
the  model  has  undergone  several  revisions  In  response  to  feedback  from  these 
applications.  This  Inherent  modifiability,  and  the  scope  of  the  model's  ap- 
plicability, are  Indicative  of  the  dynamic  nature  of  the  NAVAIR/NAVTRAEQUIPCEN 
model.  It  has  the  capacity  to  evolve  as  the  diversity  and  complexity  of  Its 
applications  Increase. 

OVERVIEW  OF  THE  NAVAIR/NAVTRAEQUIPCEN  MODEL 

4 

As  Illustrated  In  figure  1,  the  model  has  been  divided  into  five  blocks  of 
related  activities  beginning  with  Analysis  and  ending  with  Quality  Control. 
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Figure  1 . Major  Tasks 
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The  specific  tasks  to  be  accomplished  by  the  ISD  team  are  listed  in  the  order 
of  their  occurrence  within  each  major  block.  These  tasks,  and  the  rationales 
for  their  inclusion  In  the  model,  are  discussed  In  some  detail  In  Section  II 
of  this  report.  The  purpose  of  the  present  section  is  to  focus  attention  on 
the  general  features  of  the  model  so  that  the  particulars  to  follow  will  be 
seen  in  their  proper  perspective  as  the  functional  parts  of  an  integrated 
system. 

ANALYSIS.  The  Initial  effort  presented  in  the  model  Is  Analysis.  This  state 
calls  for  analysis  of  both  the  training  problem  and  the  behavior  to  be  trained. 
This  process  Involves  a thorough  study  of  the  weapon  system  under  consideration 
In  order  to  identify  the  tasks  that  must  be  performed  to  operate,  or  maintain, 
the  system.  Ultimately,  these  tasks  are  translated  into  the  component  behav- 
iors that  require  training.  The  output  from  this  stage  of  ISD  is  a complete 
specification  of  the  behavioral  objectives  that  the  new  training  program  will 
be  designed  to  achieve. 

OESIGN.  The  second  major  phase  of  activity  called  for  in  the  model  Is  Design. 
It  is  here  that  the  ISD  team  must  select  Instructional  media,  specify  the 
organization  of  lessons  and  courses,  integrate  ground  school  instruction  with 
"hands-on"  training  in  simulators  and  actual  aircraft,  and  identify  the  sources 
of  support  that  will  be  required  to  carry  out  the  indicated  Instruction. 

Initial  planning  of  Implementation  and  quality  control  procedures  also  is  begun 
at  this  point.  The  primary  output  from  the  Design  phase  of  ISD  is  a skeleton 
framework  of  the  training  program  that  will  be  required  to  achieve  the  objec- 
tives determined  during  Analysis. 

DEVELOPMENT.  It  Is  In  the  third  phase  of  ISD  that  the  program  skeleton  under- 
goes Development.  The  materials  to  be  used  in  ground  school  for  both  study 
and  evaluation  are  developed  to  meet  the  specifications  for  each  lesson  segment 
that  were  laid  out  during  the  Design  phase.  These  materials  are  then  tested 
for  effectiveness  in  small-scale  tryouts,  revised  if  necessary,  and  then  put 
into  final  production.  In  addition  to  development  of  instructional  materials 
for  students,  materials  are  also  developed  that  will  be  needed  to  train  in- 
structors in  all  facets  of  the  new  program.  The  output  from  this  stage  is  a 
complete  Instructional  package  that  is  ready  to  be  Implemented. 

IMPLEMENTATION  AND  QUALITY  CONTROL.  The  fourth  and  fifth  major  phases  of 
activity  stipulated  In  the  model  are  Implementation  and  Quality  Control.  The 
former  takes  the  ISD  team  through  the  steps  necessary  to  ensure  that  the 
training  program  will  be  successfully  put  into  effect.  The  latter  specifies 
the  procedures  to  be  used  by  the  ISD  team  to  develop  a quality  control  system 
that  will  serve  as  a management  tool  throughout  the  life  cycle  of  the  program, 
the  vehicle  through  which  the  program  will  be  self-monitoring  and  corrective. 

The  flow  of  activities  from  foalysis  to  Quality  Control  is  a logical  and 
practical  progression.  Design  of  a new  training  program  naturally  follows 
from  an  analysis  of  the  situation  for  which  the  program  is  being  designed. 
Development  of  the  program  from  the  schematic  stage  to  the  full  operational 
stage  must  be  controlled  by  design.  Likewise,  only  a fully  developed  program 
can  be  successfully  Implemented,  and  then  only  if  the  procedures,  resources, 
and  personnel  essential  to  Its  functioning  are  made  available  and  properly 
utilized.  Finally,  It  is  the  operations  of  the  functioning  program  that 
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require  quality  control  procedures  to  assure  that  it  Is  meeting  design  speci- 
fications and  that  It  will  continue  to  be  effective  In  training  the  behaviors 
Identified  during  the  analysis  phase  of  ISO. 
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SECTION  II 

DESCRIPTION  OF  THE  NAVAIR/NAVTRAEQU IPCEN  MODEL 

Whether  the  NAVAIR/NAVTRAEQU  IPCEN  model  is  applied  to  a newly  emerging 
weapons  system,  or  to  an  existing  system,  the  I SD  process  is  initiated  by  a 
documented  Indication  of  a need  for  a new  training  program  or  a revision  of  an 
existing  system.  This  initial  docunentation  identifies  the  nature,  scope,  and 
criticality  of  the  training  project.  If  it  ’s  found  to  be  of  sufficient  magni- 
tude to  warrant  further  consideration,  an  in-depth  problem  analysis  is  planned 
by  NAVTRAEQUIPCEN.  After  the  plan  has  received  the  approval  of  NAVAIRSYSCOM, 
and  the  lines  of  communication  have  been  established  among  the  various  partici- 
pating Navy  organizations,  a full-scale  problem  analysis  is  launched  under  the 
direction  of  the  ISO  group. 

The  primary  objective  of  the  problem  analysis  is  to  establish  the  precise 
nature  of  the  training  effort  that  will  be  required  to  achieve  an  effective 
instructional  program.  In  the  case  of  an  existing  training  program,  every 
aspect  of  the  system  is  examined  and  evaluated  in  order  to  identify  the  needed 
revisions  and  the  appropriate  strategies  for  making  ?hem.  The  analysis  would 
include  everything  from  instructional  syllabi,  t-aininq  materials  and  devices, 
tests  and  student  attitudes  to  management  efficiency,  and  instructional  goals 
of  the  existing  system. 

In  the  case  of  an  emerging  weapons  system,  the  analysis  is  directed  toward 
a preliminary  determination  of  the  tasks  required  to  operate  the  system,  the 
kind  of  training  program,  materials,  and  devices  that  will  optimize  student 
learning  of  the  required  tasks,  and  the  structure  of  the  Navy  organization  that 
will  be  needed  to  develop,  implement,  and  manage  the  instructional  system.  In 
the  case  of  either  an  existing  or  an  emerging  system,  the  problem  analysis 
identifies  the  personnel,  facilities,  equipment,  time,  and  costs  required  for 
development  and  implementation  of  the  needed  training  program.  This  informa- 
tion is  evaluated  relative  to  program  goals  and  assets,  and  a Problem  Analysis 
Report  (PAR)  Is  prepared.  A sample  outline  of  the  PAR  contents  for  an  existing 
system  is  shown  in  figure  2. 

If  the  projected  goals  and  requirements  of  the  needed  training  program,  as 
documented  in  the  PAR,  are  approved  by  NAVAIRSYSCOM,  then  development  of  a 
program  master  plan  (PMP)  is  undertaken  by  the  ISD  team. 

The  PMP  serves  as  a tool  for  managing  and  coordinating  the  ISD  project,  and 
it  incorporates  the  information  contained  in  the  previous  problem  analysis  to- 
gether with  other  Information  such  as  the  latest  funding  analyst'  available  and 
cost  data  from  previous  ISD  projects.  The  ISD  model  is  also  taken  into  account 
in  the  PMP  since  the  model  delineates  the  stage- by- stage  progression  of  ISD  and 
stipulates  the  products  generated  at  each  stage.  Ail  this  information  is  inte- 
nrated by  the  ISD  team  and  the  PMP  is  fonuulated. 

The  PMP  states  the  objectives  of  the  proposed  ISD  program,  and  the  proce- 
dures by  means  of  which  those  objectives  can  be  realized.  The  major  milestones 
of  the  ISD  project  are  specified,  and  a method  is  provided  for  tracking  the 
various  stages  of  the  development  process.  The  resources  and  facilities  re- 
quired for  the  ISD  project  are  inventoried  together  with  the  sources  committed 
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to  making  them  available.  The  plan  also  stipulates  the  organizational  roles 
and  responsibilities  of  participating  organizations  and  personnel,  and  provides 
a system  for  their  coordination.  The  plan  even  designates  the  specific  jobs  to 
be  performed  throughout  the  ISO  project,  and  indicates  when  each  job  Is  to  be 
performed,  and  by  whom.  The  plan  also  Identifies  potential  funding  sources, 
gives  cost  estimates  for  each  of  the  successive  stages  of  the  ISD  project, 
offers  various  procurement  strategy  options,  and  describes  the  materials 
needed  to  carry  out  appropriate  procurement  procedures.  Thus,  the  PMP  serves 
as  the  primary  management  tool  for  directing  the  development  of  a large-scale 
instructional  system.  A sample  outline  of  PMP  contents  for  an  existing  system 
Is  presented  In  figure  3. 

If  the  PMP  Is  approved  by  the  NAVAIRSYSCOM,  the  project  may  then  move  ahead 
into  the  more  detailed  task  analysis  stage  where  the  real  job  of  behavioral 
specification  and  Instructional  design  begins. 

It  Is  in  the  analysis  phase  of  ISD  where  the  techniques  of  modem  psycho- 
logy and  Instructional  technology  are  applied  most  Intensively  In  the  model. 

The  complexity  of  the  machines  that  men  must  be  trained  to  operate  and  main- 
tain, especially  airborne  weapons  systems,  requires  an  approach  to  training 
that  is  based  In  the  science  of  human  behavior  and  the  allied  technology  of 
educational  systematics. 

The  fundamental  principles  of  these  disciplines  have  been  Incorporated 
Into  the  model  In  a manner  that  Is  uniquely  adapted  to  the  needs,  resources, 
and  constraints  present  In  Naval  training  situations.  In  part,  the  unique 
applicability  of  the  model  stems  from  the  management  system  It  has  built  Into 
the  ISD  process,  and  It  Is  through  the  application  of  behavioral  and  education- 
al principles  that  the  model  becomes,  at  once,  both  an  adaptive  and  a powerful 
tool  for  solving  the  huge  and  complicated  training  problems  encountered  by  the 
modem  Navy. 

The  analysis  phase  of  ISD  begins  with  the  question,  "What  are  the  tasks 
that  must  be  performed  In  order  to  operate  the  system  in  question,  and,  under 
what  conditions  must  these  tasks  be  performed"?  The  answer  to  this  question 
is  obtained  through  a task  analysis  of  the  operational  system. 

First,  system  operation  Is  partitioned  Into  the  major  job  responsibilities 
required  for  each  mission  phase.  Then,  each  responsibility  area  is  further 
analyzed  to  Identify  the  primary  task  components  that  must  be  performed.  Each 
task  component  Is  described  In  a detailed  and  standardized  fashion.  A task 
description  states  precisely  the  conditions  under  which  the  task  Is  performed, 
the  actions  that  compose  the  performance,  and  the  particular  outcomes  of  the 
performance.  This  emphasis  on  task  identification  and  description  Is  based  on 
the  psychological  principle  that  "the  more  accurately  a behavior  can  be  speci- 
fied, the  more  efficiently  It  may  be  trained."  Thus,  task  analysis  generates 
task  specifications,  l.e.,  descriptive  statements  of  the  conditions,  actions, 
and  outcomes  that  compose  each  task  component.  By  organizing  these  task 
descriptions  according  to  responsibility  areas  and  mission  phases,  an  accurate 
picture  of  the  behavior  required  to  perform  a job  is  obtained.  In  this  case, 
the  job  description  Is  a task  listing.  A model  of  the  task  listing  procedure 
is  Illustrated  In  figure  4 and  a partial  task  list  from  the  EA-6B  program  Is 
given  as  an  example  In  figure  5. 
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The  task  analysis  and  listing  process  Is  usually  carried  out  by  a team  of 
subject  matter  experts  (SMEs),  Navy  personnel  experienced  In  the  operation  of 
the  weapon  system  under  consideration,  who  have  been  trained  In  ISO  methodology 
by  a behavioral  specialist.  Together,  the  specialist  and  SMEs  form  the  ISO 
team  responsible  for  producing  the  Initial  task  listing.  To  ensure  that  Ihe 
task  descriptions  are  accurate,  and  that  the  task  llstlnq  is  complete  and 
properly  organized,  an  Independent  group  of  SMEs  evaluates  the  work  of  the 
original  ISO  team.  They  also  estimate  the  frequency  and  criticality  of  per- 
formance of  the  Individual  tasks.  This  step  In  the  ISO  process  Is  referred  to 
as  task  validation,  and  It  may  result  In  a revision  of  the  original  listing. 

As  a means  of  standardizing  the  validation  procedure,  the  ISO  team  prepares  a 
questionnaire  which  the  Independent  group  of  SMEs  uses  to  evaluate  the  task 
list.  A sample  questionnaire  form  used  In  the  EA-6B  program  is  shown  in 
figure  6. 

Now  that  the  tasks  have  been  behavioral 1y  specified  and  functionally  organ- 
ized, the  validated  task  listing  Is  subjected  to  another  kind  of  evaluation. 
Each  task  in  the  listing  Is  examined  systematically  by  the  ISO  team  in  order 
to  determine  the  level  of  training  that  It  will  require. 

Depending  on  the  entry-level  skills  of  the  new  FRS  trainees,  as  compared 
with  the  standards  of  acceptable  performance  for  Individual  tasks,  the  ISO 
team  will  assign  each  task  to  one  of  five  different  training  categories.  These 
five  categories  are;  full-scale,  review-only,  familiarization-only,  deferred, 
and  no-training  required.  For  example.  If  a given  task  has  to  be  performed 
frequently,  and  If  Its  performance  standard  exceeds  the  entry-level  skills  of 
trainees,  the  task  would  be  scheduled  for  full-scale  FRS  training,  especially 
if  correct  performance  of  the  task  were  critical.  On  the  other  hand,  even  a 
high-frequency  task  might  be  classified  as  requiring  no  training  If  its  per- 
formance were  already  well  within  the  entry-level  skills  of  the  new  trainees. 

A model  of  this  decision-making  process  is  presented  in  figure  7. 

As  a result  of  this  careful  procedure  of  task  selection,  a major  benefit  is 
achieved.  Resources  and  time  are  not  wasted  on  unnecessary  training,  and  tasks 
that  are  essential  to  competent  performance  are  not  overlooked  In  the  training 
program.  Rather,  the  approach  taken  is  to  assign  each  task  to  just  that  level 
of  training  which  is  necessary  to  assure  that  its  performance  will  at  least 
meet  the  operational  standard. 

At  this  point  the  tasks  selected  for  FRS  training  are  reviewed  to  determine 
which  of  them  must  be  trained  in  either  real,  or  simulated,  operational  en- 
vironments. Tasks  that  require  live  enactment  or  perceptual -motor  skills  In- 
volving realistic  visual,  auditory,  motion,  etc.,  cues  are  designated  for 
"hands-on"  media  training.  This  group  of  tasks  is  further  divided  Into  two 
categories;  those  that  must  be  trained  only  in  actual  flight  conditions  (or 
only  on  actual  operational  equipment),  and  those  that  can  be  trained  under 
synthetic  operational  conditions.  It  is  the  latter  group  of  hands-on  tasks 
that  are  of  Interest  at  this  stage  of  the  ISO  process. 

Analysis  of  the  conditions  and  standards  associated  with  these  tasks  en- 
ables the  ISO  team  to  arrive  at  a definition  of  the  kinds  of  training  devices 
that  will  be  needed  for  simulated  hands-on  training.  The  attempt  here  Is  not 
only  to  select  the  most  suitable  training  medium  for  each  task,  but  to  identify 
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as  early  as  possible  those  media  which  are  expensive  and  require  time  for 
design  and  production.  This  Important  feature  of  the  model  permits  parallel 
development  of  both  the  training  program  and  the  systhetic  training  media,  thus 
avoiding  costly  delays  In  Implementation. 

The  armed  services  have  long  recognized  that  training  devices  are  less  ex- 
pensive to  use  as  training  media  than  actual  operational  systems.  The  savings 
In  personnel  time  and  fuel  consumption  alone  are  sufficient  to  warrant  their 
use  in  training  programs.  But  it  has  also  been  recognized  that  synthetic 
devices  enable  the  training  of  many  procedures  (such  as  those  involving  emer- 
gencies) that  could  not  otherwise  receive  any  degree  of  instructional  attention. 
However,  the  approach  taken  by  the  model  extends  the  advantages  of  training 
devices  even  beyond  this  level  of  utility. 

In  the  model,  behavioral  objectives  of  the  Instructional  program  control 
training  device  specification.  This  concept  Is  in  marked  contrast  with  the 
old  notion  that  a training  device  should  simulate  the  operational  system  as 
closely  as  possible.  To  the  surprise  of  many,  a training  device  optimally 
designed  to  meet  behavioral  objectives  may  involve  considerably  less  simula- 
tion than  those  produced  by  the  traditional  approach  to  training.  Furthermore, 
vrtiene  device  specifications  are  governed  by  behavioral  objectives,  the  devices 
are  not  only  more  effective  as  training  media,  but  they  may  be  less  expensive 
to  produce  and  operate.  Since  just  that  degree  of  simulation  that  is  necessary 
to  adequately  train  tasks  is  Incorporated  into  a device,  the  expense  of  un- 
necessary simulation  is  avoided.  Training  devices  designed  to  the  specifica- 
tions generated  from  the  ISD  approach  are  effective  for  minimal  cost.  It  is 
significant  that  these  devices  are  designed  to  be  an  integral  part  of  the  over- 
all training  program,  to  serve  as  Instructional  media  with  specific  training 
objectives.  This  assures  an  extensive  utilization  of  available  devices  and, 
consequently,  the  realization  of  a higher  return  on  the  Navy's  investment. 

What  we  have  seen  up  to  this  point  In  the  ISD  process  is  a systematic 

analysis  of  the  behavior  necessary  to  operate  a weapons  system.  For  each  oper- 

ator position,  the  necessary  behavior  Is  broken  down  Into  Its  component  tasks, 
and  these  tasks  are  organized  Into  its  component  tasks,  and  these  tasks  are 
organized  Into  the  functional  units  that  occur  within  each  phase  of  a mission. 
The  task  listing  Is  independently  validated,  and,  if  necessary,  then  revised. 
The  conditions,  outcome,  frequency,  standard,  and  criticality  of  performance  of 
each  task  is  specified.  These  factors,  together  with  an  assessment  of  entry- 
level  skills  of  new  trainees,  provides  the  data  needed  to  select  the  tasks  to 
receive  FRS  training  and  to  designate  the  type  and  level  of  training  required 
for  each  task.  Hands-on  tasks  that  can  be  best  trained  In  synthetic  devices 

are  Identified,  and  this  enables  an  early  specification  of  the  tralnlnq  devices 

that  will  become  an  Integral  part  of  the  training  program. 

While  the  analyses  of  behavior  carried  out  thus  far  are  adequate  for  task 
specification  and  selection,  an  even  finer  analysis  Is  necessary  to  determine 
the  nature  of  the  behavioral  objectives  that  the  training  program  must  be  de- 
signed to  achieve. 

The  distinction  between  tasks  and  behavioral  objectives  Is  fundamental  to 
the  ISD  methodology,  for  It  Is  the  objectives  that  control  the  detailed  aspects 
of  Instructional  design.  It  may  be  said  that,  whereas,  tasks  are  what  a person 
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must  do  to  operate  a system,  behavioral  objectives  are  what  a training  program 
must  achieve  to  produce  competent  task  performance. 

The  behavior  required  to  perform  a complex  task  always  contains  a number  of 
component  skills,  concepts,  decision-making  strategies,  etc.  Consequently, 
trfiole-task  performance  cannot  be  trained  satisfactorily  until  Its  more  basic 
components  have  been  learned.  For  example,  the  use  of  an  on-board  computer  to 
calculate  the  tlme-of -arrival  at  some  distant  destination  requires  that  the 
operator  not  only  possess  the  fundamental  computer  skills,  but  he  must  also 
have  knowledge  of  the  more  general  navigational  principles  and  techniques,  as 
well  as  the  basic  mathematical  skills  essential  for  numerical  computation.  Use 
of  the  on-board  computer  and  applications  of  navigational  principles  could  not 
be  taught  effectively  If  the  trainee  did  not  first  possess  the  elemental  mathe- 
matical and  computational  skills. 

So,  by  breaking  down  a task  Into  Its  fundamental  components,  and  by  com- 
paring these  with  the  skills  already  present  In  the  behavior  of  the  new 
trainees.  It  Is  possible  to  Identify  the  particular  behavioral  components  of 
a task  that  must  be  Individually  trained.  These  task  components  become  the 
behavioral  objectives  to  be  achieved  by  the  training  program. 

This  analysis  of  the  behavior  required  to  perform  a task,  however,  only 
yields  a set  of  target  behaviors  that  the  ISO  program  must  be  designed  to  pro- 
duce In  trainees  at  criterion  proficiency  levels  under  specified  conditions. 
These  target  behaviors,  or  behavioral  objectives,  may  be  many  steps  removed 
from  the  basic  entry-level  skills  of  the  trainees.  Thus,  the  behavioral  com- 
ponents Intermediate  between  entry-level  skills  and  target  behavior  also  must 
be  enumerated. 

Again,  the  driving  principle  behind  this  progressively  more  detailed  anal- 
ysis of  behavior  Is  to  make  explicit  that  which  must  be  trained.  Intermediate 
behaviors  are  prerequisites  for  target  behaviors  in  the  same  sense  that  target 
behaviors  are  prerequisites  for  task  performance.  The  relationships  among 
Intermediate  behaviors  and  target  behaviors  form  an  organizational  hierarchy 
shaped  like  a pyramid  with  the  behavioral  objectives  located  at  the  top. 
Behaviors  listed  at  each  level  of  the  objectives  hierarchy  are  always  prerequi- 
site and  essential  to  performance  of  the  behaviors  llsteaat  higher  levels  In 
the  hierarchy.  A model  of  the  hierarchy  analysis  process  Is  presented  In 
figure  8. 

Construction  of  objectives  hierarchies  is  a crucial  step  In  the  ISO  process 
for  several  reasons.  First,  the  construction  procedure  helps  to  ensure  that 
the  ISO  team  will  not  overlook  any  Important  intermediate  behaviors,  second, 
the  hierarchical  organization  of  the  Intermediate  behaviors  leading  to  each 
behavioral  objective  shows  the  sequential  order  In  which  these  behaviors  should 
be  learned,  and  third,  the  entire  subject  matter  content  of  the  Instructional 
program  Is  delineated  In  the  hierarchical  organization  of  the  behavioral  ob- 
jectives. Thus,  objectives  hierarchies  provide  a complete  picture  of  the 
diversity  of  behaviors,  and  their  Interrelationships,  that  must  be  encompassed 
by  the  Instructional  system.  A sample  objectives  hierarchy  is  illustrated  In 
figure  9. 
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Figure  8.  Analysis:  Objective  Hierarchy  Development 
{A  Hierarchy  Analysis  Model) 


Figure  9.  Analysis:  Objectives  Hierarchy  Development  (A  Sample  Hierarchy  from  EA-6B) 
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Now  that  the  question  of  what  to  teach  has  been  definitely  answered,  the 
next  question  to  be  addressed  Is  how  to  teach  It  most  effectively?  An  answer 
to  this  question  must  be  obtained  for  each  behavioral  objective,  and  each 
answer  must  Identify  both  the  method  and  medium  for  instruction  that  will 
optimize  student  learning  of  the  behavior. 

Knowing  what  to  teach  Is  usually  not  synonomous  with  knowing  how  best  to 
teach  It.  For  this  reason,  the  model  provides  the  ISO  team  with  a special 
decision-making  procedure  which  can  be  used  in  a straightforward  manner  to 
select  methods  and  media  given  certain  inputs  to  the  process.  As  Illustrated 
in  figure  10,  the  Isd  team  first  determines  the  resources  that  will  be  avail- 
able for  Instruction,  1 .e. , the  funds,  personnel,  and  facilities.  These  re- 
sources set  limits  on  the  range  of  methods  and  media  from  which  the  team  may 
select.  After  this  is  established,  the  ISD  team  looks  at  the  subject  matter- 
content  of  each  behavioral  objective  to  ascertain  the  kind  and  level  of 
learning  Involved,  the  level  of  competency  students  will  be  expected  to  obtain, 
the  kinds  of  Interactions  with  Instructors  and  materials  that  will  be  needed 
to  substantiate  and  motivate  learning,  the  specificity  and  source  of  response- 
contingent  feedback  necessary  for  self-corrective  learning,  and  the  character- 
istics of  Information  displays  essential  for  effective  presentation  of  the 
subject  matter.  When  this  information  has  been  obtained,  the  ISD  team  is 
ready  to  begin  the  decision-making  process  that  will  result  in  the  optimal 
choice  of  an  available  method  and  medium  for  each  particular  subject  matter 
under  consideration.  In  practice,  the  complexity  of  media  selection  can  be 
reduced  to  an  algorithm  such  as  that  shown  in  figure  11.  This  model  provides 
a systematic  sequence  of  steps  that  begin  with  a behavioral  objective  and  end 
with  an  appropriate  selection  of  instructional  media. 

Advances  in  the  application  of  electronics  to  the  development  of  instruc- 
tional media  has  had  a profound  influence  on  educational  technology.  This  is 
especially  evident  In  some  of  the  applications  of  computer  science.  Consider, 
for  example,  the  sophisticated  visual  and  auditory  displays  now  utilized  in 
modem  Instructional  media  and  simulation  devices,  the  advanced  computer  pro- 
grams that  allow  for  student  interation  and  provide  response-contingent  feed- 
back, and  the  many  kinds  of  information  processing  systems  that  extend  the 
limits  on  man's  memory  and  thinking  capabilities. 

No  longer  must  the  quality  of  Instruction  be  Invested  primarily  in  the 
expertise,  ability  to  communicate,  >nd  motivating  influence  of  the  traditional 
Instructor.  The  diversity  of  methods  and  media  existing  today  permit  the  in- 
structional designer  to  choose  the  one  that  Is  most  appropriate  for  each  be- 
havior to  be  trained.  Furthermore,  the  new  methods  and  media  rely  less  on  the 
Instructor  operating  as  a lecturer,  motivator,  and  evaluator.  In  ISD  programs. 
Instructional  presentation  and  evaluation  Is  Individualized,  and  motivation  is 
maintained  through  reinforcing  contingencies  built  Into  the  progression  of 
events  each  student  encounters  as  he  moves  through  the  program.  This  shifts 
the  responsibilities  of  Instructors  and  students  from  those  of  lecturer  and 
pupil  to  those  of  guide  and  learner.  The  respective  role  of  each  is  enhanced, 
as  Is  their  motivation  to  perform  well.  The  new  responsibilities  of  instruc- 
tors and  students  In  the  training  program  designed  according  to  the  NAVAIR/ 
NAVTRAEQUIPCEN  model  are  largely  due  to  the  methods  and  media  selected  for 
use,  but  they  are  also  dependent  upon  the  way  Instructional  courses  are  devel- 
oped and  sequenced. 
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The  sequence  of  instructional  experiences  through  which  the  students  move 
in  an  ISO  training  program  is  stipulated  in  a course  syl Tabus,  a kind  of  map  in 
which  behavioral  objectives  are  organized  into  lesson  sequences.  A given 
course  is  divided  into  major  units  of  instruction,  and  each  unit  consists  of  a 
number  of  lessons  which  are  further  broken  down  into  individual  segments. 

These  divisions  of  a course  are  sequenced  to  lead  onto  each  other  such  that  the 
student  is  moved  in  a steady  path  from  lower  to  higher  order  knowledges  and 
skills. 


The  progression  of  learning  experiences  is  arranged  so  that  the  student's 
knowledge  and  skills  build  in  increments  small  enough  to  never  overextend  the 
students'  capability,  but  large  enough  to  ensure  an  optimal  rate  of  progression 
from  entry  levels  to  the  more  complex  job  levels  of  performance.  This  kind  of 
arrangement  is  designed  to  prevent  failure  at  one  point  In  a course  due  to  in- 
complete experience  gained  at  previous  points  in  the  course.  At  each  stage, 
the  student  has  all  the  experience  he  needs  to  proceed  successfully  to  the 
next  s tage . 

Instruction  programmed  in  this  fashion  has  proved  to  be  a far  more  effi- 
cient approach  than  the  traditional  one  because  it  not  only  reduces  failure  to 
a minimum,  but  it  also  produces  a high  level  of  student  motivation.  When 
effort-to-leam  results  in  success,  this  feedback  motivates  the  student  to 
continue.  Thus,  the  sequential  ordering  of  learning  experiences  into  well- 
designed  course  syllabi  is  a crucial  aspect  of  the  ISO  process.  It  is  at  the 
stage  of  syllabi  development  that  the  ISD  team  moves  trom  behavioral  analysis 
to  instructional  design.  Incorporating  the  Information  contained  in  the  objec- 
tives hierarchies  into  a framework  designed  to  maximize  learning. 

The  transformation  from  objectives  hierarchies  to  course  syllabi  is  nearly 
as  complicated  as  It  is  Important,  and  ISO  team  members  require  some  training 
and  expert  guidance  in  this  process.  Essentially,  the  question  they  must 
answer  Is  "into  what  sequences  must  the  behavioral  objectives  be  ordered?", 
and  this  question  must  be  answered  for  each  behavioral  objective. 

Usually,  the  position  of  a behavioral  objective  in  its  hierarchy  will 
determine  Its  sequential  position  in  a course  syllabus.  Objectives  located  at 
the  bottom  of  a hierarchy  are  more  elementary  than  those  above  them  and,  thus, 
should  be  taught  first.  However,  if  all  low-level  objectives  were  taught  be- 
fore moving  up  the  hierarchy,  the  student  would  tend  to  become  bored  and 
possibly  even  forget  some  of  the  Information  he  has  already  learned  because  he 
would  not  have  had  an  opportunity  to  apply  it.  So,  this  problem  is  avoided  by 
introducing  hands-on  experience  Into  the  syllabus  as  soon  as  possible.  Gener- 
ally, this  can  be  accomplished  by  limiting  successive  sets  of  objectives  to 
vertical  legs  of  the  hierarchy.  In  other  words,  the  designer  starts  with  the 
lowest  objective  on  one  vertical  leg  and  moves  up  to  the  point  where  a hands-on 
objective  is  encountered.  The  latter  may  be  anything  from  a familiarization 
exercise  In  a trainer  to  an  actual  flight  in  an  aircraft.  This  type  of  se- 
quence cycles  the  student  from  purely  ground  school  type  situations  to  equip- 
ment exercises,  then  back  to  ground  school  for  more  basics  followed  by  further 
equipment  exercises.  A flow  diagram  of  the  course  organization  and  sequencing 
process  is  shown  in  figure  1?. 


L 


NAVTRAEQUIPCEN  IH-304 


* 


t Ufi«im4n«  th* 
nunthar  ol  flight! 
m»  h tiudfiM  will  I HI 

glVtU! 

t 


2 Identify  *M  htndi 
On  iwilmmiw  « nh|#« 
ItiM  wlurh  «li>>uld 
profit  "MHy 
v m Ih*  A V 

flight! 


3 liimip  and 
uM|M*iMr>  prifotmani  k 
turn  into  flight! 


" r 

4 Identity  *11  WSI 
ol*|»«*  tnr*l  rMiltUMV 
to  ui|i|hm  I not  mat 
operation!  to*  •*<  h 
flight 


S Identity  *||  WSI 
oh|*«tiv*\  rt*«ling  with 
•m«rg«iH  y oiietalinm 
ptot  *ntoi*\ 


f 


t 

h t«roii|i  .mil 
H<|tlflll  * nil  W*.  | 

l»l*t*!  1 1 V • ! 


1 - -4 


1 111  — ~ • -1 

t 

t Identify  jll  othei 

» Identify  ail  othet 

handy-on  normal 

Itandvon  »m*ig«iKy 

o(»arat«m«  objact»»ei 

1 ' 

operation!  nig*  fiv*% 

1 ' 

I 

H («imi|i  .lull 

III  1 .i.m-o  in  t 

.til  i'imim ui’iii  v 

««l»iM.itmn\  ftaniai! 

"1”  t.ltlOO!  IMIIH  U 

T 


i 


(tump  amt 
\ tl*IH  f all  ttotfy 


17  iiMni|i  li'!\iiti!  into 

Oltlts 


Figure  12.  Design:  Course  Organization  and  Sequencing  (The 
Organization  and  Sequencing  Process) 
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This  type  of  cycling  maintains  the  integrity  of  the  objectives  hierarchy 
while  allowing  the  student  to  practice  his  newly  acquired  skills  as  soon  as 
possible.  In  this  way,  a small  portion  of  the  syllabus  is  encountered, 
learned,  and  practiced  before  the  student  moves  on.  Those  objectives  re- 
quiring actual  flight  in  the  aircraft  are  preceded  by  objectives  of  a more 
elemental  nature  that  are  practiced  in  a trainer  or  simulator.  The  objectives 
prerequisite  to  trainer  exercises  are  still  more  elemental  and  they  constitute 
the  content  core  of  ground  school  instruction.  Typically,  the  student  would 
go  through  several  evolutions  of  the  cycle  between  ground  school  and  trainer 
exercises  before  cycling  up  to  an  aircraft  flight,  and  the  objectives  to  be 
accomplished  in  the  first  aircraft  flight  would  be  less  difficult  than  those 
scheduled  for  later  flights.  At  each  stage  In  the  syllabus,  the  student  is 
prepared  to  advance  to  the  next  stage. 

Segment-by-segment,  the  ISO  team  organizes  related  behavioral  objectives 
into  lessons.  Elemental  lessons  are  placed  ahead  of  more  difficult  ones  In 
the  syllabus,  and  a test  is  scheduled  for  each  lesson.  The  lessons  that  per- 
tain to  a given  subject  matter  are  organized  into  instructional  units,  and  an 
equipment  exercise  is  scheduled  at  the  end  of  each  unit.  Finally,  the  various 
units  of  instruction  are  structured  into  a framework  that  forms  the  course 
syllabus. 

Not  only  do  course  syllabi  serve  as  maps  of  the  instructional  sequences 
for  the  entire  training  program,  but  they  also  provide  the  ISO  team  with  a 
sufficiently  complete  picture  of  the  program  to  permit  an  accurate  analysis  of 
the  new  program's  training  support  requirements.  Consequently,  it  is  at  this 
stage  that  the  ISO  team  estimates  the  total  personnel,  equipment,  services, 
materials,  and  facilities  that  will  be  required  to  complete  the  development  of 
the  training  program,  and  to  implement  and  maintain  it  throughout  its  life 
cycle. 

For  example,  the  trainer  specifications  generated  earlier  during  task  se- 
lection are  now  reexamined  to  determine  if  each  trainer  will  encompass  a 
sufficient  number  of  objectives  to  be  utilized  fully,  and  if  each  is  the  least 
expensive  device  that  can  be  effective  in  training  the  specified  objectives. 
Similarly,  the  classroom  media  selected  earlier  are  now  evaluated  to  determine 
which  media  are  most  feasible  for  presenting  the  Information  within  each 
lesson.  The  answers  to  these  questions  enable  the  ISD  team  to  estimate  the 
purchase,  modification,  production,  and  operating  costs  required  for  the 
various  trainers  and  Instructional  media.  However,  in  order  to  estimate 
personnel,  services,  and  facilities  requirements,  the  ISD  team  must  construct 
a daily  time-based  class  schedule  for  all  courses  in  the  training  proqram. 

From  this,  the  team  can  establish  the  student  flow  throughout,  the  proqram  and 
the  support  requirements  on  a lesson-by-lesson  basis.  The  major  steps  in- 
volved in  training,  support  requirements  analysis  are  illustrated  in  figure 
13. 

Training  support  requirements  analysis  provides  the  information  needed  by 
planners  and  managers  to  assure  the  availability  of  critical  support  resources 
needed  for  satisfactory  completion  of  the  training  program  and  its  ultimate 
implementation  and  maintenance.  By  basing  this  analysis  on  completed  course 
syllabi,  a major  source  of  potential  program  failure  is  avoided,  i.e.,  failure 
due  to  the  unavailability  of  some  key  Instructional  resource  around  which  much 
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of  the  training  program  has  been  designed  and  upon  which  program  Implementa- 
tion Is  dependent.  Rigorous  support  requirements  planning  at  this  stage  In 
the  ISO  process  ensures  that  only  those  resources  available  to  the  program  are 
Incorporated  Into  Its  design. 

Now  that  the  skeleton  of  the  program  has  been  built  and  the  resources 
needed  to  support  It  have  been  established,  the  time  has  come  for  the  ISO  team 
to  put  some  meat  on  the  bones,  l.e.,  to  write  the  lesson  specifications.  The 
critical  subject  content  and  teaching  strategy  must  now  "be  specified  for  each 
behavioral  objective  In  the  course  syllabus.  For  example.  If,  in  order  to 
achieve  a particular  objective,  the  student  must  learn  a rule,  a definition,  a 
fact,  or  a procedure,  the  ISD  team  must  state  It  explicitly.  Likewise,  If  a 
particular  kind  of  explanation,  or  mnemonic  aid,  will  facilitate  learning  of 
the  objective,  this  also  Is  specified.  In  addition,  the  ISD  team  determines 
the  various  kinds  of  Illustrative  examples,  practice  problems,  and  test  Items 
that  will  best  exemplify  and  evaluate  the  behavior  called  for  in  the  objec- 
tives. The  team  also  stipulates  the  graphic  illustrations  to  be  Included  In 
the  Instructional  material  for  each  objective.  After  this  has  been  completed 
for  all  the  objectives  covered  In  a given  lesson,  a lesson  format  guide  Is 
prepared  which  explains  how  the  material  In  each  segment  will  be  organized, 
and  how  the  Individual  segments  will  be  tied  together  to  form  the  lesson.  A 
sample  format  guide  outline  for  one  lesson  segment  is  presented  In  figure  14. 
The  output  from  this  stage  of  the  ISD  process  Isa  set  of  tight  guidelines 
that  will  be  used  to  control  the  organization  of  the  detailed  subject  matter 
content. 

The  guidelines  help  to  avoid  the  kind  of  instruction  which  gives  either 
too  much,  or  too  little,  attention  to  certain  materials,  misses  the  point,  or 
buries  It  In  a mass  of  detail,  or  ignores  fundamental  considerations  of 
teaching  strategy.  Proper  lesson  specification  ensures  that  the  principles  to 
be  learned  in  any  group  of  behavioral  objectives  determine  both  the  kind  and 
degree  of  detail  given,  as  well  as  the  strategy  that  Is  chosen  to  teach  It. 

The  lesson  specifications  and  format  guides  provide  the  groundwork  for 
actual  lesson  authoring.  Working  from  the  specifications  of  lesson  contents 
and  format,  the  ISD  team  write-out  paper  and  pencil  versions  of  the  final  In- 
structional materials.  After  review  and  editing,  these  materials  go  Into 
prototype  production  for  use  In  small-scale  tryouts  with  real  students. 
Usually, tnese  prototype  Instructional  materials  are  scheduled  for  tryouts  as 
they  are  being  developed.  By  means  of  these  tryouts  the  ISD  team  Is  able  to 
determine  whether  the  materials  are  actually  effective  In  bringing  about  the 
desired  learning,  and  to  establish  whether  the  materials  are  palatable  to  the 
students.  Instructional  materials  that  are  found  to  be  weak  in  either  respect 
are  revised  and.  If  necessary,  tried  out  again. 

This  procedure  maximizes  the  probability  that  the  Instructional  materials 
will  be  successful  once  the  training  program  is  implemented.  After  the 
materials  have  undergone  their  final  revision,  they  are  then  scheduled  for 
final  production.  The  activities  of  the  ISD  team  during  this  evaluation 
process  are  diagramed  In  figure  15,  parts  1 through  4.  The  finished  product 
Is  a package  that  comes  as  close  to  guaranteeing  effective  Instruction  as  can 
* - be  provided  by  modern  technology.  However,  the  NAVAIR/NAVTRAEQUIPCEN  model 

4#  recognizes  that.  In  order  for  this  training  package  to  remain  effective 
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throughout  Its  life  cycle,  It  will  not  only  have  to  be  managed  properly,  but 
It  will  need  to  be  amenable  to  continuous  evaluation  and  updating. 

Once  the  Instructional  media  and  materials  have  entered  the  final  produc- 
tion phase,  the  ISO  team  can  turn  its  attention  to  developing  a plan  that  will 
control  Implementation,  operation,  and  long  term  evaluation  of  the  training 
program.  It  Is  at  this  point  that  the  basic  Instructional  management  system 
Is  established,  as  Illustrated  In  figure  16.  This  system  defines  the  rotes 
of  instructional  personnel,  the  student  management  procedures,  and  the  proce- 
dures for  resource  allocation  and  scheduling. 

Built  Into  the  management  process  is  a quality  control  system  designed  to 
continuously  assess  the  effectiveness  and  palatabllity  of  the  Instructional 
materials,  as  well  as  the  instructional  management  system  Itself.  An  Illus- 
tration of  this  system  Is  presented  In  figure  17.  The  objective  of  this 
quality  control  system  is  to  provide  a mechanism  for  Identifying  those  pieces 
of  Instruction  that  require  modification,  and  for  assuring  that  the  needed 
modifications  will  be  made  in  a timely  and  smooth  manner. 

This  Is  the  final  task  of  the  ISO  team.  Upon  its  completion,  the  team 
will  have  developed,  designed,  and  produced  a superior  instructional  system, 
one  which  can  be  Implemented  and  managed  efficiently  for  the  lifetime  of  the 
program.  The  built-in  quality  control  system  assures  that  the  program  will 
always  accomplish  what  It  was  designed  to  do  and  that  Its  materials  will  be 
revised  to  reflect  the  changes  in  conditions  and  constraints  that  will  proba- 
bly lie  In  the  future.  Such  a program  should  be  as  fresh  and  effective  at  the 
end  of  Its  tenure  as  It  was  at  the  beginning. 
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